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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 
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DETAILED ACTION 
Drawings 

1. New corrected drawings in compliance with 37 CFR 1.121(d) are required in this 
application because the current, informal drawings are not of sufficient quality and are 
not in accordance with 37 CFR 1 .84. Applicant is advised to employ the services of a 
competent patent draftsperson outside the Office, as the U.S. Patent and Trademark 
Office no longer prepares new drawings. The corrected drawings are required in reply to 
the Office action to avoid abandonment of the application. The requirement for 
corrected drawings will not be held in abeyance. 

2. Figure 1 should be designated by a legend such as -Prior Art— because only 
that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in 
compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid 
abandonment of the application. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1.121(d)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they do not include the following reference sign(s) mentioned in the 
description: 100, 102, 105, 115, 120, 125, 130, 135, 620, and 930. Corrected drawing 
sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to 
avoid abandonment of the application. Any amended replacement drawing sheet should 
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include all of the figures appearing on the immediate prior version of the sheet, even if 
only one figure is being amended. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1.84(c)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

5. Claim 1, 3, 5, 10, 11, and 13 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Yeo (US Pat No. 6,71 1,741), cited by examiner. 

With regards to claim 1 , the claimed method for providing video data is met as 
follows: The claimed step of opening a main connection for receiving transmissions of a 
data flow, is met by data path 324 (Figure 3), as described in column 3, lines 48-50, 
wherein Yeo teaches a server sending source video frames through data path 324 and 
temporal snapshots through data path 322 to the client 102. The claimed step of 
opening a second connection for transmission of at least one look-x data stream 
comprising data from said data flow is met by data path 322 (Figure 3), as described in 
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column 3, lines 48-50, discussed above. The claimed step of indexing at least one point 
of the look-x data stream to at least one corresponding point in the data flow is met by 
the relation between the temporal snapshots and the continuous video frames. Column 
4, lines 52-58 teach a relation between the snapshots and the continuous video, Yeo 
states that the display requests the snapshot's corresponding segment of source video 
frames 106 from the server 100. This correspondence meets the claimed indexing step. 
The claimed step of providing control of a playback position of the data flow based on 
the indexed points in the look-x data stream is met by column 4, lines 52-58, wherein 
Yeo teaches the ability for a user to control the playback and select the source frames 
to view based on the temporal snapshots. 

With regards to claim 3, the claimed step of displaying at least one of a skip 
forward and a skip backward button configured to step a play position of the data flow to 
a position corresponding to a respective one of a next and a previous look-x data points 
relative to the current play position of the data flow is met by column 5, lines 1-3, 
wherein Yeo teaches the use of options within the media player 612 to step to the 
previous frame or the next frame, so as to allow the user to step through the playback 
sequence. 

With regards to claim 5, the claimed data flow being a video and look-x points 
being frames of the data flow and being retrieved from the main connection or the 
second connection is met by column 3, lines 48-50, wherein Yeo discloses source video 
frames 106 (the video) being sent on the main data path 324, and temporal snapshots 
(the look-x points/frames) being sent on a second data path 322. 
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With regards to claim 10, the claimed device for client side video indexing is met 
as follows: The claimed video player is met by media player 612 (Figure 6a). The 
claimed main data stream for receiving transmissions of a data flow is met by stream 
324 and the connection to network 104 (Figure 3 and as described in column 3, lines 
48-50). The claimed look-x data stream connection for receiving transmissions of the 
data flow is met by stream 322 and the connection to network 104 (Figure 3, and as 
described in column 3, lines 48-50). The claimed controller for indexing at least one 
point of the look-x data stream to a corresponding at least one point in the data flow is 
met by the relation between the temporal snapshots and the continuous video frames. 
Column 4, lines 52-58 teach a relation between the snapshots and the continuous 
video, Yeo states that the display requests the snapshot's corresponding segment of 
source video frames 106 from the server 100. This correspondence meets the claimed 
controller for indexing. The claimed display for displaying at least one of the indexed 
look-x points is met by video display 216 (Figure 2) and application window 600, which 
displays temporal snapshots 602-610 (look-x points) and media player 612 (Figure 6a). 

With regards to claim 1 1 , the claimed skip forward and skip backwards buttons 
that each step a play position of the data stream to a respective one of a next and a 
previous look-x points relative to the current play position of the data stream are met by 
column 5, lines 1-3, wherein Yeo teaches the use of options within the media player 612 
to step to the previous frame or the next frame, so as to allow the user to step through 
the playback sequence. 
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With regards to claim 13, the claimed select button for providing a user the 
capability to select at least one of the indexed points enabling display of the data flow to 
begin and the selected indexed point is met by column 4, lines 52-58, wherein Yeo 
teaches the ability for a user to control the playback and select the source frames to 
view based on the temporal snapshots. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claim 2, 4, 6, 7-9, 12, and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yeo (US Pat. No. 6,71 1 ,741 ), cited by examiner, in view of Ceccarelli 
(US Pat. No. 6,222,532), cited by examiner. 

With regards to claim 2, Yeo teaches all of that which is discussed above with 
regards to claim 1 . Yeo does not expressly disclose the step of displaying a timeline 
corresponding to the indexed look-x points, the timeline having at least one of the 
indexed look-x points displayed so as to reference a position on the timeline. Ceccarelli 
discloses a system with a timeline, which makes reference to a group of keyframes 
(indexed look-x points). Figure 4 is shown as a display to the user, with a timeline of the 
whole video at top, and a dark bar section, which indicates the section of keyframes 
displayed (144-148). Figure 4 and column 3, line 65 - column 4, line 47 give a detailed 
description of the "timeline" aspect of the invention. It would have been obvious to one 
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of ordinary skill in the art at the time of the invention to include the timeline as taught by 
Ceccarelli into the system as taught by Yeo in order to allow the user to get an overall 
feel for the progression of the video and frame location of the video presentation. Yeo 
does not provide specifics for the media player 612; however, a timeline would be an 
obvious addition to a typical media player. 

With regards to claim 4, Yeo teaches all of that which is discussed above with 
regards to claim 1 . Yeo does not expressly disclose the steps of displaying a timeline 
having representations of the indexed points. Yeo does however disclose the steps of 
selecting at least one of the indexed points, and displaying the data flow at a point 
beginning with the selected indexed point. Both of these steps are met by column 4, 
lines 52-58, wherein Yeo teaches the ability for a user to control the playback and select 
the source frames to view based on the selection of temporal snapshots. As stated 
before, Yeo does not expressly disclose the step of displaying a timeline with indexed 
points. Ceccarelli does disclose a system with a timeline, which makes reference to a 
group of keyframes (indexed look-x points). Figure 4 is shown as a display to the user, 
with a timeline of the whole video at top, and a dark bar section, which indicates the 
section of keyframes displayed (144-148). Figure 4 and column 3, line 65 - column 4, 
line 47 give a detailed description of the "timeline" aspect of the invention. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to include the 
timeline as taught by Ceccarelli into the system as taught by Yeo in order to allow the 
user to get an overall feel for the progression of the video and frame location of the 
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video presentation. Yeo does not provide specifics for the media player 61 2; however, 
a timeline would be an obvious addition to a typical media player. 

With regards to claim 6, Yeo teaches all of that which is discussed above with 
regards to claim 1 . Yeo does not expressly disclose that the second connection is a low 
resolution connection relative to the main connection. Ceccarelli discloses a system in 
which the provider (or server) can label keyframes before transmission to the client, in 
order to allow the client easy access to the chosen keyframes (column 1 , lines 18-27). 
While Ceccarelli does not specifically state the use of two connections to the server 
(one for main video and one for the keyframes), he does, however, make a realization 
that "video distortions in relatively smallish keyframes have been experienced as 
tolerable, [and that] if a particular keyframe is enlarged, extra measures should be taken 
for picture improvement" (column 2, lines 4-7). This indicates an inherent 
understanding that the keyframes are often times transmitted as lower resolution 
images and will need some processing in order to display them at higher resolution 
requirements. Therefore, it can be understood that the lower resolution keyframes 
could be used and downloaded over a second lower resolution connection (the 
connection taught by Yeo), It would have been obvious to one of ordinary skill in the art 
at the time of the invention to combine the low resolution connection, as described by 
Ceccarelli, into the system of Yeo, in order to allow for more bandwidth conservation 
and the ability for the user to download more information regarding the keyframes and 
look-x points and not take up massive amounts of bandwidth in the process. 
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With regards to claim 7, Yeo teaches all of that which is discussed above with 
regards to claim 1 . Yeo does not expressly disclose any of the steps of selecting, 
displaying, or updating the indexed look-x points. Yeo simply discussed the TSG 300 
(column 3, lines 18-50), which serves to select and send the temporal snapshots to the 
client. Ceccarelli goes into detail about how the keyframes are selected, displayed, and 
updated. The claimed step of selecting a predetermined number of indexed look-x 
points is met by Figure 5, which shows a display of nine keyframes for selection by a 
user. Nine keyframes is more or less just a suggestion and could presumably be 
modified to select any number of keyframes as predetermined by the system. The 
claimed step of displaying the predetermined number of indexed points to provide 
reference for a playback control mechanism is met by Figure 5 again, which shows the 
typical display to the user with nine keyframes being selectable and referencing different 
portions within the video to allow for playback control. The claimed step of updating the 
selected predetermined number of index look-x points based on an update criteria is 
met by column 4, line 67 - column 5, line 3, wherein Ceccarelli teaches that the display 
is updated as the video progresses. The update criteria are based upon a distance 
from the current position. At the time of the invention, it would have been obvious to 
one or ordinary skill in the art to include the selecting, displaying, and updating steps as 
taught by Ceccarelli into the system as taught by Yeo, in order to allow the user to see 
pertinent keyframes and allow them to view keyframes in the near future and recent 
past. 
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With regards to claim 8, Yeo and Ceccarelli teach all of that which is discussed 
above with regards to claims 1 and 7. Yeo does not expressly disclose any step of 
selecting a predetermined number of look-x points such that each of the look-x points is 
within a predetermined distance of a first play position of the data flow. Ceccarelli 
inherently teaches that the number of keyframes on the display fall within a 
predetermined distance of the play position of the video. It is disclosed on column 4, 
lines 53-58, that the keyframes are located within seconds of each other, with the center 
keyframe being the currently displayed video segment. Therefore, Ceccarelli inherently 
shows that the keyframes that are currently displayed are within a predetermined time, 
and therefore distance of the play position. At the time of the invention, it would have 
been obvious to one of ordinary skill in the art to combine the inherent teachings of 
Ceccarelli, as discussed here, with the system as taught by Yeo, in order to allow the 
user to see pertinent keyframes and allow them to view keyframes in the near future 
and recent past. 

With regards to claim 9, Yeo and Ceccarelli teach all of that which is discussed 
above with regards to claims 1 and 7, Yeo does not expressly disclose that the 
aforementioned update criteria comprise a change of the playback position a 
predetermined amount from the first play position during the selection step. Ceccarelli 
discloses a way of updating the keyframes (column 4, line 67 - column 5, line 3) and 
the idea of having the currently viewed video segment and its related keyframe in the 
center and past and future keyframes to the left and right, respectively. In other words, 
Ceccarelli discloses a dynamic display, which updates according to the currently viewed 
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video segment. At the time of the invention, it would have been obvious to one or 
ordinary skill in the art to implement the updating criteria of Ceccarelli with the system of 
Yeo, in order to allow for a dynamically updated display of pertinent keyframes in the 
near future and recent past. 

With regards to claim 12, Yeo teaches all of that which is discussed above with 
regards to claim 10. Yeo does not teach the display displaying a timeline referenced to 
the data flow and at least one of the indexed look-x points, the indexed look-x points 
each displayed so as to reference a position on the timeline. Ceccarelli discloses a 
system with a timeline, which makes reference to a group of keyframes (indexed look-x 
points). Figure 4 is shown as a display to the user, with a timeline of the whole video at 
top, and a dark bar section, which indicates the section of keyframes displayed (144- 
148). Figure 4 and column 3, line 65 - column 4, line 47 give a detailed description of 
the "timeline" aspect of the invention. It would have been obvious to one of ordinary 
skill in the art at the time of the invention to include the timeline as taught by Ceccarelli 
into the system as taught by Yeo in order to allow the user to get an overall feel for the 
progression of the video and frame location of the video presentation. Yeo does not 
provide specifics for the media player 612; however, a timeline would be an obvious 
addition to a typical media player. 

With regards to claim 14, Yeo teaches all of that which is discussed above with 
regards to claim 10. Yeo does not expressly disclose that the look-x data stream 
connection is a low resolution data stream relative to the main data stream connection. 
Ceccarelli discloses a system in which the provider (or server) can label keyframes 
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before transmission to the client, in order to allow the client easy access to the chosen 
keyframes (column 1, lines 18-27). While Ceccarelli does not specifically state the use 
of two connections to the server (one for main video and one for the keyframes), he 
does, however, make a realization that "video distortions in relatively smallish 
keyframes have been experienced as tolerable, [and that] if a particular keyframe is 
enlarged, extra measures should be taken for picture improvement" (column 2, lines 4- 
7). This indicates an inherent understanding that the keyframes are often times 
transmitted as lower resolution images and will need some processing in order to 
display them at higher resolution requirements. Therefore, it can be understood that the 
lower resolution keyframes could be used and downloaded over a second lower 
resolution connection (the connection taught by Yeo). It would have been obvious to 
one of ordinary skill in the art at the time of the invention to combine the low resolution 
connection, as described by Ceccarelli, into the system of Yeo, in order to allow for 
more bandwidth conservation and the ability for the user to download more information 
regarding the keyframes and look-x points and not take up massive amounts of 
bandwidth in the process. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Steele (US Pat. No. 5,884,056) discloses a system and method for browsing 
videos on the web. He utilizes a timeline with selectable points within the video object. 
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Bronson (US Pat. No. 5,135,655) discloses a system and method for indexing 
and retrieving audio-video data through the use of generated indexing words and 
pictures. No network is used, however, two connections (one for video data, and one 
for index data) are provided to the display device. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael R Shannon whose telephone number is 703- 
305-6955. The examiner can normally be reached on M-F 7:30-5:00, alternate Friday's 
off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Miller can be reached on 703-305-4795. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 


Michael R Shannon 

Examiner 

Art Unit 2614 


Michael R Shannon 
September 22, 2004 
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